Certain visual applications require separating the foreground from the background in images of a video feed in order to put the foreground subject into another context. Classical applications of such a separation and recombination are the representation of the TV weather person in front of a virtual meteorological map, commercials that appear to be filmed in an exotic scenery, super heroes in movies that appear to be present in artificial sceneries or flying in the sky. Such representations have in common that their human subjects are recorded in a studio and that the background is replaced by a new one. The procedure is relatively easy if the original background can be controlled, or if it is fully known to the video processing unit. This is the case for the examples above, which are recorded in a specially designed studio.
The present invention deals with the problems that arise when the above mentioned technique has to be used with an arbitrary background, which has to be detected or learned by the video processing unit.